Abstract

Single-photon avalanche diodes (SPADs) are novel image sensors that record photons at extremely high sensitivity. To reduce both the required sensor area for readout circuits and the data throughput for SPAD array, in this paper, we propose a snapshot compressive sensing single-photon avalanche diode (CS-SPAD) sensor which can realize on-chip snapshot-type spatial compressive imaging in a compact form. Taking advantage of the digital counting nature of SPAD sensing, we propose to design the circuit connection between the sensing unit and the readout electronics for compressive sensing. To process the compressively sensed data, we propose a convolution neural-network-based algorithm dubbed CSSPAD-Net which could realize both high-fidelity scene reconstruction and classification. To demonstrate our method, we design and fabricate a CS-SPAD sensor chip, build a prototype imaging system, and demonstrate the proposed on-chip snapshot compressive sensing method on the MINIST dataset and real handwritten digital images, with both qualitative and quantitative results.

Method

In order to realize efficient SPAD array sensing, we design a novel compressive sensing SPAD array which can directly record the compressively sensed data. In the decoding process, we propose a neural network designed to directly process the compressively sensed data, which can reconstruct the scene and realize classification.

Snapshot Compressed Imaging Chip

We propose to implement compressive sensing in a block-wise way. The basic compressed coding unit directly records the readout of the sensor after exposure by linking readout electronics (with memory) to each pixel, as shown in Figure 1(a)-(b). , the connection settings within each block of pixels are shown in Figure 1(c)-(f).

Figure 1. (a) Classic sensor array (b), basic compressed sensing imaging unit of CS-SPAD, (c)-(f) four different CS connection settings.

Information Processing Architecture Based on Convolution Neural Network

We propose CSSPAD-Net to realize multi-task processing upon the compressed measurements from our chip, realizing both image reconstruction and classification. The proposed CSSPAD-Net includes a reconstruction branch and a classification branch, and the features of the reconstruction branch are fused into the classification branch through the bringe operation.

Figure 2. (a) Overview of CSSPAD-Net, (b) the structure of residual block (Resblock), (c) the structure of residual dense block (RDB).

Experiments

CS-SPAD Sensor Chip

We designed a 32 × 32 CS-SPAD sensor chip to realize on-chip snapshot-type spatial compressive imaging. The system mainly includes three parts: a 32 × 32 SPAD detector array, readout circuits, and address decoding circuits. The 32 × 32 SPAD detector circuit includes the SPAD detector, the corresponding gating quenching circuit, and the logic circuit required for compressive sensing.

Figure 3. (a) CS SPAD chip overview, (b) the diagram of CS-SPAD sensor chip design, (c) single pixel layout.

Optical System

We build a prototype imaging system based on the proposed CS-SPAD sensor. From left to right are the target scene, the lens used for focusing, the CS-SPAD sensor, and the host computer for controlling the automatic execution of the system.

Figure 4. (a) Overview of the prototype imaging system, (b) the detail of camera lens and CS SPAD chip, (c) the detail of peripheral circuits.

Simulation Results

For quantitative evaluation, with the MNIST test dataset, we display the images on the screen, capture the compressed data with the CS-SPAD sensor, reconstruct the image with the CSSPAD-Net, and calculate the reconstruction metrics with the reconstructed image and the corresponding projected image.

Figure 5. Reconstructed results by CSSPAD-Net.

Real Handwritten Data Experiment

Furthermore, as shown in Figure 6, we handwrite more digits, not from MNIST, for verification of our entire system. We use the same model trained on the MNIST dataset to classify and reconstruct the real handwritten digits. The visual display of the reconstructed results are shown in Figure 7, as shown, our methods could realize elegant reconstruction with the compressed results captured by the CS-SPAD sensor.

Figure 6. Example of real handwritten digits.

Figure 7. Real handwritten digits: reconstructed results.

More Details

• File
• Paper

Bibtex

@article{qiu2023chip,

title={On-Chip Compressive Sensing with a Single-Photon Avalanche Diode Array},

author={Qiu, Chenxi and Wang, Peng and Kong, Xiangshun and Yan, Feng and Mao, Cheng and Yue, Tao and Hu, Xuemei},

journal={Sensors},

volume={23},

number={9},

pages={4417},

year={2023},

publisher={MDPI},

url={https://www.mdpi.com/1424-8220/23/9/4417}

}